Necessary conditions for linear nonco operative 
Af-player delta differential games on time scales 11 

Natalia Martins Delfim F. M. Torres 

Department of Mathematics 
University of Aveiro 
3810-193 Aveiro, Portugal 

{natalia, delfim}Oua.pt 



Abstract 

We present necessary conditions for linear noncooperative iV-player 
delta dynamic games on a generic time scale. Necessary conditions for 
an open-loop Nash-equilibrium and for a memoryless perfect state Nash- 
equilibrium are proved. 

Mathematics Subject Classification 2000: 49N70, 91A10, 39A10. 

Keywords: time scales, delta differential games, calculus of variations on time 
scales. 



1 Introduction 

In 1988 Stephan Hilgcr developed in his PhD thesis 8 the theory of time scales. 
The set R of real numbers and the set Z of integers are (trivial) examples of 
time scales. When a result is proved in a general time scale T, one unifies both 
continuous and discrete analysis. Moreover, since there are infinitely many 
other time scales, a much more general result is proved. For this reason, one 
can say that the two main features of the theory of time scales are unification 
and extension. 

Differential game theory is a relatively new area of Mathematics, initiated in 
the fifties of the XX century with the works of Rufus Isaacs, that founds many 
applications in different fields such as economics, politics, and artificial intelli- 
gence. The theory has been studied in the context of classical Mathematics, in 
discrete or continuous time [TJ [Tl] . We trust that it is also possible (with some 
advantages) to present (delta) differential games in the generic context of time 
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scales. To the best of the authors knowledge, this paper represents the first 
attempt to provide a delta differential game theory on a generic time scale. 

The paper is organized as follows. In Section [5] we review some basic def- 
initions and results from the calculus on time scales. In Section [3J necessary 
conditions for a weak local minimizer of a Lagrange problem on time scales 
(Theorem are presented, while Section [5] recalls a result that guarantees the 
uniqueness of the forward solution for a special initial valued problem on time 
scales (Theorem [7]). In Section [5] we introduce the definition of a iV-player delta 
differential game, the notion of Nash-equilibrium, and two types of informa- 
tion structure in a game: open-loop and memoryless perfect state. The main 
results of the paper appear then in Section where necessary conditions for 
a linear open-loop Nash-equilibrium and for a linear memoryless perfect state 
Nash-equilibrium are proved. 

2 Calculus on time scales 

We briefly present here the necessary concepts and results from the theory of 
time scales (cf. 0IH[9] and references therein). As usual, R, Z and N denote, 
respectively, the set of real, integer, and natural numbers. 

A time scale T is an arbitrary nonempty closed subset of R. Thus, R, Z and 
N are examples of times scales. Other examples of times scales can be hZ, for 
some h > 0, [1,4] (JN, or t ne Cantor set. We assume that a time scale T has 
the topology that it inherits from the real numbers with the standard topology. 

The forward jump operator a : T — > T is defined by 

a(t) = inf {s S T : s > t} 

if t ^ supT and cr(supT) = supT. The backward jump operator p : T — > T is 
defined by 

p(t) = sup{s e T : s < t} 

if t ^ inf T and p(MT) = inf T. 

A point t S T is called right-dense, right- scattered, left-dense or left-scattered 
if a(t) — t, ait) > t, p(t) — t or p(t) < t, respectively. 

The graininess function p : T — > [0, oo) is defined by 

pit) = a(t) - t, for all t e T. 

For a given instant t, p(t) measures the distance of t to its right neighbor. 

It is clear that when T = R, then for any t G R, a(t) = t = p(t) and p = 0. 
When T = Z, for any t e Z, a(t) = t + 1, p(t) = t - I, and p = 1. 

In order to introduce the definition of delta derivative, we define the set 

T k := {t G T : t is nonmaximal or left-dense} . 

Thus, T fc is obtained fr om T by removing its maximal point if this point exists 
and is left-scattered. 
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We say that a function / : T — ► R is delta differ entiable at t 6 T k if there is 
a number f A (t) such that for all e > there exists a neighborhood U of i (i.e., 
+ <5[nT for some J > 0) such that 

\f(a(t)) - f(s) - f A (t)(a(t) - s)\ < e\a(t) - s\, for all s G U. 

We call f A {t) the rfeZia derivative of / at t. Moreover, we say that / is delta 
differ entiable on T fc provided f A {f) exists for all t G T fe . 

We note that when T = R, then / : R -> R is delta differentiable at t G R if 
and only if 



r(t) = lim ■ 



+t i — s 

exists, i.e. if and only if / is differentiable in the ordinary sense at t. When 
T = Z, then / : Z — > R is always delta differentiable at t G Z and 

/ A (*)= /(g( 2j" / /(t) =/(* + !) -/(*)■ 
o-(t) - t 

Hence, for T — 1 the delta derivative of /, / A , coincides with the usual forward 
difference A/. 

It is clear that if / is constant, then f A = 0; if /(<) = kt for some constant 
k, then f A = k. 

For delta differentiable functions / and g, the next formulas hold: 

f (*) = /(*) + M*)/ A (*), 

(/5) A W = / A (i)3 ff W + /W.9 A W 
= / A WffW + rW.9 A W, 

where we abbreviate / o a by f a . 

Delta derivatives of higher order are defined in the standard way: we define 
the r th - delta derivative (r G N) of / to be the function f A : T* — ► R, provided 

/ A ' _1 is delta differentiable on T fcr := (t*^ 1 ) . 

The class of continuous functions on T is too small for a convenient theory 
of integration. For our purposes, it is enough to define the notion of integral in 
the class of rd-continuous functions. For a more general theory of integration 
on time scales, we refer the reader to [3]. A function / : T — > R is called rd- 
continuous if it is continuous at right-dense points and if its left-sided limit exists 
(finite) at left-dense points. For T = R rd-continuity coincides with continuity. 

We denote the set of all rd-continuous functions / : T — > R by C r( j(T, R), 
and the set of all delta differentiable functions with rd-continuous derivative by 
C^T.R). 

It can be shown that every rd-continuous function / possess an antideriva- 
tive, i.e. there exists a function F with F A = f, and in this case the delta 
integral is defined by 

[ f(t)At := F(b) - F(a) for all a, b G T. 

J a 
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This integral has the following property: 

x(t) 



f(r)AT = l x(t)f(t). 

Let a, b G T and / € C r( j(T,R). It is easy to prove that 

1. for T = R, J Q b f{t)At = j f(t)dt, where the integral on the right hand 
side is the usual Riemann integral; 

2. for T = Z, J b a f(t)At = E t t« /(*) if a < 6, f* f(t)At = if a = b, and 

/ 6 /(*)At=-E^6/(*) if«>&- 

Next, we present the integration by parts formulas: if a, b G T and /, p G 
C rd (T,R), then 

1- £ = [(/5)(t)]'=a~/ Q b / A (%WAt; 
2. J a b /(% A (t)At = [(/ 5 )(t)£* - ti f A (t)g(a(t))At. 

Similarly to classical calculus, we say that / : T — > R" is a rd- continuous 
function if each component of /, /j : T — > R, is a rd-continuous function. The 
set of all such functions is denoted by C rd (T, R n ). The set C* d (T, R n ) is defined 
in the usual way. 

3 Lagrange problem on time scales 

Let a,b G T such that a < b. In what follows we denote by [a, b] the set 
{t G T : a < t < b}. Consider the following Lagrange problem with delta 
differential side condition: 



■/[*(•), u(-)] = f L{t,x(t),u(t))At- 

x A (t) = <p{t,x(t),u(t)), teT k 
x(a) = x a , 



(1) 



where we assume that 

• L : T x R™ x R m -> R, ip : T x R" x R m -» R", (2, it) -» L(t,x,u) and 
(x, it) — > a;, u) are C 1 -functions of x and it for each f ; 

• x(-) G C£ d (T,R") and u(-) G C rd (T,R m ), m < n; 

• for each control function u(-) there exists a unique forward solution x(-) 
of the initial value problem x A (t) = ip(t,x(t),u(t)), x(a) — x a Q 



1 In the linear case, conditions guaranteing the existence and uniqueness of forward solutions 
are easy to obtain [2] - cf. Section [4] 
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We borrow from [7] the definition of admissible pair and the definition of 
weak local minimizer for problem |T]). The reader interested in the calculus of 
variations on time scales is referred to [5J [5] and references therein. 

Definition 1. The pair (ar*(-), u*(-)) is said to be admissible for problem (QP if 

1. £*(•) G C^(T,R n ) and u.(-) £ C rd (T, R m ); 

2. %*(t) = (f(t, x*(t), and x* (a) = x a . 

In order to introduce the notion of weak local minimizer for problem ([1]) , we 
define the following norms in C rd (T,R n ) and Cy d (T, R n ): 

Mvlloo :=Bup||y(t) ||, yeC rd (T,R n ), 
teT 

and 

||«||i,co := sup || *(t) || + sup || z A (t) || , z 6 Cy d (T,R n ) , 
where || • || denotes the Euclidean norm in R™. 

Definition 2. An admissible pair (x*(-), * s sa * c ' ^° ^ e a weak local mini- 

mizer for problem FJJ) if there exists r G R + such that 

J[x*(-),u*(-)} < ■/[*(•), «(•)] 

/or a/Z admissible pairs satisfying 

\\ x - ||i )O0 + || u - ||oo< r. 

Theorem [3] gives necessary conditions for a pair (£*(•), «*(•)) to be a weak 
local minimizer of the Lagrange problem ([1]) . 

Theorem 3 ([7]). If (<£*(•), is a weaA; ZocaZ minimizer of problem fip, £/ien 
i/iere exists a multiplier '■ T ~~ * R n i/iai is de/ia differ entiable on T k , such 

that 

x A (t) = H^(t,x,{t),u,(t),^(t)) , 
<P*(t) = -H x (t,x*{t),u*(t),tf(t)), 

Mb) = 0, 

for all t G T fc , where the Hamiltonian function Ti is defined by 

H(t, x, u, ip a ) '■= L(t, x, u) + t[) a ■ ip(t, x, u) . (3) 

Remark 4. From definition f3J) of the Hamiltonian function Ti., it follows that 
the first condition in (TJP holds for any admissible pair (#*(•), «*(•)) of problem 

w> 

Remark 5. For i/ie time sca/e T = R, Theorem^ is a particular case of Pon- 
tryagin's Maximum Principle J12f . 
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4 Linear systems on times scales 



Let us consider the following initial value problem on a time scale T: 

x A (t) = Ax(t) + f(t), 
x(a) = x a , 

where A is a constant n x n matrix, / : T — > K.™ is a rd-continuous function, 
a £ T and x a £ K". 

Similar to control theory [2], in dynamic game theory we are interested in 
forward solutions. The purpose of this section is to present conditions assuring 
problem ^ to have a unique forward solution. 

Proposition 6 ( 3 ]). The initial value problem 

x A {t) = Ax{t), 
x(a) = x a , 

where A is a constant n x n matrix, a £ T and x a £ R n , has a unique forward 
solution. 

The matrix exponential function (also known as the fundamental matrix 
solution) at a for the matrix A, is defined as the unique forward solution of the 
matrix differential equation 

X A (t) = AX(t), 

with the initial condition X(a) = I, where / denotes the n x n identity matrix. 
Its value at t is denoted by e A (t, a) . 

Theorem 7 (cf. [5]). The initial value problem ^ has a unique forward solu- 
tion of the form 

x(t) = e A (t, a)x a + / e A (t, a(s))f(s)As. 



5 TV-player delta differential games 

The (classical) term "TV-player dynamic game" is applied to a group of problems 
in applied mathematics that possess certain characteristics related with conflict 
problems. The main ingredients in a iV-player dynamic game are the players, the 
control variables, the state variables, and the cost functionals/functions. The 
relation between state and control variables is given by a differential/difference 
equation. Two types of games can be considered: cooperative or noncooperative 
games. In this paper we shall restrict ourselves to noncooperative games. In a 
noncooperative game the players act independently in the pursuit of their own 
best interest, each player desiring to attain the smallest possible cost. 

Following Jank [10] (see also pQ), we introduce the notion of a TV-player delta 
differential game (noncooperative dynamic game in the context of time scales) . 
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Definition 8. Let N G N. We say that 

Tat = (T, X, C/j, C7i, /, a; OJ »/*, J l )i=i,2,...,N 
is a N -player delta differential game if: 

1. T is a closed nonempty interval off (T is called the time horizon ); 

2. X is a finite dimensional Euclidean space (X is called the state or phase 
space); 

3. Ui is a finite dimensional Euclidean space (Ui is called the control value 
space of the i-th player); 

4- Ui is a subset of a set of mappings 

{7*17* : T x V{X) C/J 

(Y is called a strategy of the i-th player while Oi is called the set of 
possible strategies of the i-th player); 

5. x is a mapping from T to X (x is called the state variable,); 

6. rf : T — > V(X) is a mapping with the property 

rf{t) C {x{s) \ a<s<t} 
(rf is called the information structure of the i-th player); 

7. Ui — {l l {-,rj l (-)) | 7* € Oi} (Ui is called the control space or decision set 
of the i-th player); 

8. f : T x X xUi x ■ ■ ■ x Un — > X is a mapping that describes an outer force 
acting on the system by the delta differential equation 

x A (t) = f(t,x(t),u 1 (t),...,u N (t)) 

with the initial condition x(a) — x a G X (u l G Ui is called the control 
function or control variable of the i-th player); 

9. J 1 is a mapping from U\ x • • • x Un to R (J 1 is called the cost functional 
of the i-th player). 

Remark 9. For each i — 1,2, . . . , N, the controls = 7 J (-,f7*(-)) G Ui are 
built up from chosen strategies -f l with a specific information structure rf . 

Remark 10. The cost functional J 1 of the i-th player, i G {1,2,..., N}, de- 
pends on the controls of all the N players. 

Definition 11. A N -tuple of control functions u(-) = . . . ,u N ()) is called 

an admissible control for the system Tn if u is a rd- continuous function and 
there exists a unique forward solution x G C^ d (T,X) of the initial value problem 
x A (t) = /(MWVW, • • • ,u N {t)), x(a)=x a . 
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We shall consider the following equilibrium concept in a TV-player delta dif- 
ferential game. 

Definition 12. A decision N -tuple u* = (ul, . . . , u# ) € U\ x • • • x Um is called 
a Nash-equilibrium if for all i = 1, 2, . . . , N , 

J\<, . . . , U f ) < J 4 (ui, . . . , <"\ U \ . . . , u? ) 

/or a/Z £ 

The meaning of the Nash-equilibrium is the following: if the «-th player 
unilaterally change the strategy ul, then his cost will increase. 

We remark that the information available to the players is an important 
aspect of the game. In this paper we consider open-loop and memoryless perfect 
state information structures. 

Definition 13. Let Tn be a N -player delta differential game. We say that Tn 
has 

1. Open-loop (OL) information structure if rf(t) = {x a },t £ T, for all i £ 
{1,2,..., N}; 

2. Memoryless perfect state (MPS) information structure ifrf{t) — {x a , x(t)}, 
t e T, for all i e {1,2, . . .,N}. 

Therefore, in the open-loop information structure each player know only the 
initial position x a , while in the memoryless perfect state information structure 
each player know the phase state x(t) at each time instant t as well as the initial 
position x a - 

Definition 14. We say that a control u l (t) := (t , rf (t)) is: (i) an OL control, 
if the information structure rf is OL; (ii) a MPS control, if the information 
structure rf is MPS. 

6 Main results 

Using Theorem [31 we deduce necessary conditions for OL and MPS Nash- 
equilibrium of iV-player games with linear delta differential equations. We deal 
with cost delta-integral functionals of the following type: 

J\u\... lU N )= f '^(MW,^),...,^)^, (5) 

J a 

where we suppose the following (from now on, T denotes TP [a, b]): 

• for each i = 1,2, .. .,7V, L l : TxR™ xR™ 1 x. . .xR m « — ► R is a C 1 -function 
of x and u — (it 1 , . . . , u ) for each t; max{mi, . . . , tun} < n; 

• for each i = 1, 2, . . . , N, v? S Uj, is admissible; 
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• x is the forward solution of the delta differentiable initial value problem 

(x A (t) = Ax(t) + + • • • + B N u N (t) , 

|x(a) = a; a , 

where A is a constant n x n matrix, and for each i — 1, 2, . . . , N, B % is a 
constant n x m, matrix. 

Remark 15. 5mce eac/i u 4 G C rd (T,R mi ), and therefore B 1 ^ H h S^it^ 

is a rd- continuous function, by Theorem [?] i/ie initial value problem |6p /las a 
unique forward solution. 

Theorem 16 (Necessary conditions for a linear OL Nash-equilibrium) . Let Tjv 
be a N -player delta differential game, where the cost functional of the i-th player 
is given by {3Jj. If the decision N -tuple (ul, . . . , u^) is an OL Nash-equilibrium 
of Tjv, and if x* is the associated trajectory of the state, then there exist delta 
differentiable functions ip 1 : T — > W l on T , i = 1, 2, . . . , N, such that for 

H l (t, x,u\..., u N , (?/>T) := V(t, x,u\..., u N )+(ip i ) a ■(Ax+B 1 u 1 +- ■ -+B N u N ) 

one has: 

1. x?(t) = Hy r (t, z.(t), uj(t), uf (t), (*)); 

2. X* \Cbj — X a i 

3. (#) A (t) - -H*(t, x*(t), ul(t), (^)'(t)); 
* #(&) = 0; 

5. • ■ .,«?(*), ty'H*)) = 0; 

/or all i = 1,2,..., N and t e T k . 

Proof. Suppose that the decision TV-tuple (ul , . . . , ) is an OL Nash-equilibrium 
of and x* is the associated trajectory of the state. For each i = 1, . . . , N 
consider the functional 

J 1 -Mi^R 

defined by 

J\u l ) :=J i {ul,...y- x ,u i ,v}+ x ,...,u^). 

Define also 

f(x, u 1 ,..., u N ) := Ax + 5 V + ■■■ + B N u N . 
Since (it* , . . . , ) is a Nash-equilibrium, then for all u l G Ui 
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Therefore, (a;* (-) , (•)) is a weak local minimizer of the problem 

J l (u l ) ► min, 

x A (t) = f(x(t),u i (t)) ) (7) 
x(a) = x a , 

where f(x,u l ) := f(x,ul, . . . , it* -1 , u\ ...,uf). Define 

W*(t, x, u*, (VT) := a, «i, • • • , <\ u\ < + \ W>T) , 

with H % the Hamiltonian defined in the statement of the theorem. Applying 
Theorem [3] to TC 1 (note that / is a C 1 -function of x and u % for each t), we 
conclude that there exists a function ip l : T — > M™ that is delta differentiable on 
T fc , such that = 0, 

= Ax*(t) + fl x ui(t) + • • • + BXW + • • • + B N u?(t) 
= Hy y (t, x.(t), ul(t),..., (^T(i)) , 

(#) A (t) = -wU*,^(*),<(*),(^) CT W) 

= - J ffi(t,^(t),^(i) ) ..., u f(t),(^rw) 5 

and 7£ 4 (i,s.(t),ti*(t),(^r(t)) - ff;(M*(t),«Ki),->«fW,(f)1<)) = 0. 

□ 

Remark 17. Notice that conditions 1 and 2 of Theorem 1 1 61 simply assert that 
X %S Qj solution of the initial value problem 

Theorem 18 (Necessary conditions for a linear MPS Nash-equilibrium) . Let 
Tjsr be a N -player delta differential game, where the cost functional of the i- 
th player is given by p]). If the decision N -tuple (ul(-) , . . . , (■)) , given by 
(7*('j x a, £*(•)); ■ ■ ■ 1 7* (') x a, x *{')))> * s a MPS Nash- equilibrium of Tpj and if 
£*(•) is the associated trajectory of the state, then there exist delta differentiable 
functions ip* : T -> E™ on T k , i = 1, 2, . . . , N, such that for 

H\t, x,u\...,u N , (?/>T) := L% x,u\..., u N )+(^) a ■(Ax+B 1 u 1 +- ■ -+B N u N ) 

one has: 

1. xf(t) = H l m „{t,x*{t),u\{t), u?(t), WY{t)); 

2. (y(i) — x a , 

3. (V/) A (t) = -K{t, x*(t), ul(t), . . . , uf (t), m a {t)) 

N 

- J2 fli J (t,s.(<),«i(*) > --.,«i) r (*).(^r(*))-72x(*^a,a:.(«)) 

3=1 

(7*2, denotes the partial derivative of 7* lujt/i respect to x); 
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I ^(b) = 0; 

5. flj,(f, x*(t), ui(t), . . . , u?(t), (VT (*)) = 0; 
/or all i = 1,2,..., N and te T k . 

Proof. Differently from Theorem 1161 now the controls u 1 depend explicitly on 
the state variable x. Suppose that the decision TV-tuple (ul, . . . ,u^) is a MPS 
Nash-equilibrium of Tn, and x* is the associated state trajectory. Fix i S 
{1, 2, . . . , N}. The same reasoning as used in the proof of Theorem [TBI permit 
to conclude that (x*(-), (•)) is a weak local minimizer of problem ([7]), where 
we suppose now that the controls are MPS. In the following we will prove that 

min J* (it*) = min J*(u*) . (8) 

all admissible MPS controls all admissible OL controls 

An OL control can be considered as a MPS control, so it is clear that 

min J\ ul ) < mm J l (u l ) . 

all admissible MPS controls all admissible OL controls 

Since u\{t) — 7* (t, x a , x{t)) is a MPS Nash-equilibrium control, by the assump- 
tion of admissibility, the equations x A (t) = f{x(t),'y\(t, x a ,x(t))) and x(a) = x a 
define a unique trajectory x*. With this trajectory wc define now the OL control 

<(*) == li(t,x a ,x*(t))- 

Notice that 

J\vl)= min J l {u l ), 

all admissible MPS controls 

and hence equality ([8|) holds. Observe also that (x*(-), i s a weak local 

minimizer of problem Q. Applying Theorem [3] to W, we conclude that there 
exists a delta differentiable function ip l : T — > M. n on T k such that 

= -Hj c (t,x*(t),ut(t),...,u?(t),(i> i r(t)) 

N 

- J2 K 3 (*, «.(t), «i(t), . . . , uf (t), ty*)* (t)) • 7 ^(t, a:,(t)) . 
3=1 

The other conditions are obtained in a similar way. □ 
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